Content information processing device and method for displaying identified object images within video content

ABSTRACT

An information processing apparatus comprising that includes a reproduction unit to reproduce video content comprising a plurality of frames; a memory to store a table including object identification information identifying an object image, and frame identification information identifying a frame of the plurality of frames that includes the object image; and a processor to extract the frame including the object image from the video content and generate display data of a reduced image corresponding to the frame for display.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 14/276,599,filed May 13, 2014, which is a continuation of U.S. Ser. No. 12/940,512,filed Nov. 5, 2010, now U.S. Pat. No. 8,787,618, issued Jul. 22, 2014,and claims the benefit of priority of Provisional Application Ser. No.61/323,564, filed Apr. 13, 2010. The entire contents of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a content information processingdevice, a content information processing method, a content informationprocessing program, and a personal digital assistant, which play atleast content information to be played along a time axis such as with amoving image, for example.

2. Description of the Related Art

In recent years, there have come into wide-spread use contentinformation processing devices which include a large-capacity storagedevice, play various types of content information, for example, such asa moving image stored in the storage device.

Also, for example, with Japanese Unexamined Patent ApplicationPublication No. 2008-98726 (FIG. 2), an audio-visual device has beendescribed wherein information indicating content features correspondingto a content playback position is obtained, and a progress bar to whichthe feature information thereof is added is displayed according to acontent playback position. According to the technique described in thepresent patent publication, information used for telling what is in acontent is displayed within one screen in a compact manner, whereby auser can readily tell what is in the content, and can select a desiredcontent. In particular, according to the technique described in thepresent patent publication, for example, characteristic pictures withincontent are displayed by being arrayed in a primary relation, wherebythe user can tell the playback positions of these pictures within thecontent in a list form.

Also, with Japanese Unexamined Patent Application Publication No.2008-17041 (FIG. 13), a moving image content playback device has beendescribed wherein, for each frame, a plurality of information relatingto a face image portion which appears within the frame thereof arecorrelated, and when a user selects a desired face image portion, asmall image of a scene where the face image portion thereof appears isdisplayed. This moving image content playback device described thereinis configured so as to present, to the user, persons in a movingcontent, and the positions thereof, in a suitable and understandablemanner.

Also, with Japanese Unexamined Patent Application Publication No.2001-211408 (FIG. 24), an image playback device and so forth have beendescribed wherein at the time of a fast forward mode, multiplethumbnails are displayed at the bottom of a screen, and also a playbackbar is displayed. With the image playback device described therein, afast forward function can be realized wherein a desired image can besearched out of a great number of recorded images over a short amount oftime.

SUMMARY OF THE INVENTION

Incidentally, in the case that a desired scene is found out of movingimage contents, in general, a user operates a moving image playbackdevice to perform fast forwarding/fast rewinding playback, and finds adesired scene while viewing fast forwarding/fast rewinding playbackimages displayed on a display screen.

However, in the case of fast forwarding/fast rewinding, images areplayed in order in this fast forwarding direction or fast rewindingdirection, and accordingly, the user often has to wait for a long timefor the display to reach a desired scene. Also, it is generally hard toview a display image at the time of fast forwarding/fast rewindingplayback, and accordingly, users often overlook desired scenes. Thus, inthe event of overlooking a desired scene, the user has to performfinding of a desired scene again from the beginning.

Note combining the techniques described in Japanese Unexamined PatentApplication Publication Nos. 2008-98726 (FIGS. 2) and 2001-211408 (FIG.24) can be conceived to relieve the load of the user to some extent, butit should be said that this is still insufficient for finding a scenethat the user desires readily and in a sure manner.

Particularly, for example, in the case that a moving image content isplayed and displayed at a personal digital assistant which a usercarries around, the user is not able to constantly pay particularattention to the screen of this personal digital assistant, and in sucha usage situation it is not easy to find a desired scene.

It has been found to be desirable to provide a content informationprocessing device, a content information processing method, a contentinformation processing program, and a personal digital assistant, whichallow the user to find a desired scene in a small amount of time in areadily and sure manner while playing a moving image content.

An information processing apparatus according to an embodiment of thepresent invention includes a reproduction unit to reproduce videocontent comprising a plurality of frames; a memory to store a tableincluding object identification information identifying an object image,and frame identification information identifying a frame of theplurality of frames that includes the object image; and a processor toextract the frame including the object image from the video content andgenerate display data of a reduced image corresponding to the frame fordisplay.

The information processing apparatus may also include a user interfaceto receive a user selection corresponding to the object image.

The processor of the information processing apparatus may be configuredto detect the object image corresponding to the object identificationinformation from the frame when the video content is reproduced.

The processor of the information processing apparatus may be configuredto generate a progress bar displaying a total playback time and aplayback progress of the video content.

The processor of the information processing apparatus may be configuredto populate the progress bar with a predetermined mark correlating thereduced image with the progress bar based on playback time informationof the frame.

The information processing apparatus may also include a user interfaceconfigured to receive a selection corresponding to the reduced image,wherein the processor is configured to move a reproduction position ofthe video content to a reproduction position of the frame correspondingto the selected reduced image.

The processor of the information processing apparatus may be configuredto detect the object image from each of the plurality of frames of thevideo content during reproduction to generate the table including objectidentification information identifying the object image, and the frameidentification information identifying each of the plurality of framesthat include the object image.

The information processing apparatus may further include a userinterface configured to display an object detection window displaying aplurality of object images for selection, and receive an inputcorresponding to one of the plurality of displayed object images,wherein the processor is configured to extract a frame from theplurality of frames including the selected object image, and generatethe display data of said reduced image from the extracted frame.

The object identification information in the table may be a keywordrepresenting an object image, and the processor is configured to detectan object image corresponding to a keyword registered in the table fromat least one of the plurality of frames during reproduction of the videocontent.

The processor of the information processing apparatus may be configuredto extract, of a plurality of frames including the object image, asingle frame representative of the plurality of frames over apredetermined time period to generate the display data.

The processor of the information processing apparatus may be configuredto extract, of a plurality of frames including the object image, a frameat a time of a scene change occurring from the video content to generatethe display data.

Accordingly to another exemplary embodiment, according to the presentinvention, includes a computer-readable recording medium includingcomputer program instructions, which when executed by an informationprocessing apparatus, cause the information processing apparatus toperform a method comprising: reproducing video content comprising aplurality of frames; storing a table including object identificationinformation identifying an object image, and frame identificationinformation identifying a frame of the plurality of frames that includesthe object image; extracting the frame including the object image fromthe video content; and generating display data of a reduced imagecorresponding to the frame for display

The method may further include receiving, at a user interface of theinformation processing apparatus, a selection corresponding to theobject image.

The method may further include detecting the object image correspondingto the object identification information from the frame when the videocontent is reproduced.

The method may further include generating and displaying a progress barindicating a total playback time and a playback progress of the videocontent.

The method may further include populating the progress bar with apredetermined mark correlating the reduced image with the progress barbased on playback time information of the frame.

The method may further include receiving a selection corresponding tothe reduced image; and moving a reproduction position of the videocontent to a reproduction position of the frame corresponding to theselected reduced image.

The method may further include displaying an object detection windowincluding a plurality of object images for selection; receiving an inputcorresponding to one of the plurality of displayed object images;extracting a frame from the plurality of frames including the selectedobject image; and generating the display data of said reduced image fromthe extracted frame.

The method may further include extracting, of a plurality of framesincluding the object image, a single frame representative of theplurality of frames over a predetermined time period to generate thedisplay data.

The method may further include extracting, of a plurality of framesincluding the object image, a frame at a time of a scene changeoccurring from the video content to generate the display data.

According to the above configurations, while a moving image content isplayed, in the case that there are object images registered in the tablewithin a frame of the playback moving image thereof, the object imagesthereof can be selected. Further, in the case that a desired objectimage has been selected out of these object images, the reduced image ofeach frame including this object image selected out of this moving imagecontent is displayed on the screen.

With the above configurations, in the event that an object has beenselected within a playback image of a moving image content, the reducedimage of a frame including the object image thereof is displayed on thescreen. Thus, a user can find a desired scene from a moving imagecontent being played in a small amount of time readily and in a suremanner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a schematic internalconfiguration example of a portable telephone terminal according to anembodiment of the present invention;

FIG. 2 is a diagram to be used for detailed description of a movingimage content file and an object search table;

FIG. 3 is a diagram illustrating a basic display example of a displayscreen at the time of a moving image content being played;

FIG. 4 is a diagram illustrating a specific screen display example inthe case that a moving image content registered in the object searchtable is played and displayed;

FIG. 5 is a diagram illustrating each frame example searched from amoving imager content with reference to the object search table based onan object image in FIG. 4;

FIG. 6 is a diagram to be used for description of a reduced imageexample of each frame searched by the object image of a person's facebeing selected;

FIG. 7 is a diagram to be used for description of a reduced imageexample of each frame searched by the object image of an airplane beingselected;

FIG. 8 is a diagram to be used for description of a reduced imageexample of each frame searched by the two object images of a person'sface and an airplane being selected out of playback images of a movingimage content;

FIG. 9 is a diagram illustrating an example of each object image to berecognized as a different object image by an image recognition andanalysis technique according to the related art;

FIG. 10 is a diagram to be used for description of processing forrecognizing as the same object image even though the size or a componentdiffers with the embodiment;

FIG. 11 is a diagram illustrating each object image to be recognized asthe same object image even though the size or a component differs withthe embodiment;

FIG. 12 is a diagram to be used for description of each object imagerecognized as the same object image even though the size or a componentdiffers with the embodiment; and

FIG. 13 is a flowchart illustrating the time of a control and arithmeticunit executing detection of an object, creation of an object searchtable, generation and display of a reduced image and a progress barusing a content information processing program.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

An exemplary embodiment of the present invention will be described belowwith reference to the drawings.

Note that, with the present embodiment, a portable telephone terminalcapable of playing content data to be played along the time axis such asa moving image or the like will be exemplified as an example to whichthe present invention is applied, but it goes without saying thatcontents that will be described here are consistently an example, andthe present invention is not restricted to this.

FIG. 1 illustrates a schematic internal configuration example of aportable telephone terminal according to the present embodiment.

In FIG. 1, a communication antenna 12 is, for example, a built-inantenna, and performs transmission/reception of signal waves forcommunication such as speech, e-mail, or the like. A communicationcircuit 11 performs frequency conversion, modulation, demodulation, orthe like of transmission/reception signals.

A speaker unit 20 is made up of a speaker for receivers, a ringer(ringtone), a speaker for output such as reproduced music or the like,and peripheral circuits thereof, provided to the portable telephoneterminal. The speaker unit 20 subjects the digital audio data suppliedfrom the control and arithmetic unit 10 to digital/analog conversion,amplifies this, converts the audio signal after amplification thereofinto an acoustic wave, and output this into the air.

A microphone unit 21 is made up of a microphone for transmitting speechas well as external audio sound collection, and peripheral circuitsthereof. The microphone unit 21 converts the acoustic wave into an audiosignal, amplifies the input audio signal thereof, subjects this toanalog/digital conversion, and transmits the audio data afteranalog/digital conversion thereof to the control and arithmetic unit 10.

A display unit 13 is made up of, for example, a liquid crystal panel,organic EL (Electro Luminescent) display, or the like, and peripheralcircuits thereof. This display unit 13 displays various types ofcharacters or message such as e-mail or the like on the above display,or displays various types of menu screens, a still image, a movingimage, or the like, using the image signal supplied from the control andarithmetic unit 10.

The operating unit 14 is made up of various types of keys, a touchpanel, and peripheral circuits thereof, provided above the casing of theportable telephone terminal according to the present embodiment. Thisoperating unit 14 converts operating input from the user into anelectric signal, amplifies the operating input signal thereof, subjectsthis to analog/digital conversion, and transmits the operating inputdata after analog/digital conversion thereof to the control andarithmetic unit 10.

An external interface unit 22 is made up of, for example, a connectorfor cable connection, and an interface circuit for external datacommunication at the time of performing data communication through acable, or a slot for external memory in/from which external memory isinserted/detached, and an interface circuit for this external memorydata communication, or the like. With the present embodiment, examplesof data to be exchanged via this external interface unit 22 include thedata of a moving image content. The moving image content data receivedvia this external interface unit 22 is stored in a memory unit 15 underthe control of the control and arithmetic unit 10. Note that in the casethat the moving image content data obtained through this externalinterface unit 22 is data not subjected to compression encoding, anarrangement may be made wherein after temporarily being stored in thememory unit 15, the data thereof is read out, subjected to, for example,compression encoding or the like by the control and arithmetic unit 10as background processing, and restored in the memory unit 15.

A short-distance wireless communication unit 23 is made up of acommunication antenna for short-distance wireless radio waves such as awireless LAN, Bluetooth (registered trademark), or the like, and ashort-distance wireless communication circuit. With the presentembodiment, examples of data to be exchanged via this short-distancewireless communication unit 23 include the data of a moving imagecontent. The moving image content data received by this short-distancewireless communication unit 23 is stored in the memory unit 15 under thecontrol by the control and arithmetic unit 10. Note that in the casethat the moving image content data received through this short-distancewireless communication unit 23 is data not subjected to compressionencoding, an arrangement may be made wherein after temporarily beingstored in the memory unit 15, the data thereof is read out, subjectedto, for example, compression encoding or the like by the control andarithmetic unit 10 as background processing, and restored in the memoryunit 15.

Note that, with the present embodiment, the data of the above movingimage content may be data received through the communication antenna 12and communication circuit 11. In this case, this received moving imagecontent data is stored in the memory unit 15 under the control of thecontrol and arithmetic unit 10. Also, at this time, in the case thatthis received moving image content data is data not subjected tocompression encoding, an arrangement may be made wherein aftertemporarily being stored in the memory unit 15, the data thereof is readout, subjected to, for example, compression encoding or the like by thecontrol and arithmetic unit 10 as background processing, and restored inthe memory unit 15.

A television reception unit 24 is made up of a reception antenna and atuner for so-called digital television. This television reception unit24 receives desired digital television broadcast airwaves, demodulatesthe received digital television broadcast airwaves, and then transmitsthese to the display unit 13. Thus, a digital television broadcastpicture is displayed on the display screen of the display unit 13. Also,the digital television broadcast data received and demodulated at thistelevision reception unit 24 may be stored in the memory unit 15, i.e.,may be recorded. In the case of recording a digital televisionbroadcast, it is desirable that this digital television broadcast datais stored in the memory unit 15 after being subjected to compressionencoding or the like by the control and arithmetic unit 10.

A camera unit 25 is made up an imaging device for taking a still imageor moving image, an optical system, and so forth, and peripheralcircuits thereof. The still image data and moving image data taken bythis camera unit 25 is transmitted to the control and arithmetic unit 10to be subjected to compression encoding or the like, and is then storedin the memory unit 15.

The control and arithmetic unit 10 is made up of a CPU (CentralProcessing Unit), and performs communication control at thecommunication circuit 11, display control of the display unit 13,terminal operation control according to the operating input data of theoperating unit 14, each operation control of the external interface unit22, short-distance wireless communication unit 23, television receptionunit 24, camera unit 25, and so forth, and performs arithmeticcomputation along therewith. Also, the control and arithmetic unit 10performs execution of various types of control programs or applicationprograms stored in the memory unit 15, and various types of dataprocessing along therewith.

Now, particularly in the case of the present embodiment, the control andarithmetic unit 10 is configured so as to execute each function of theimage processing function unit 10 a, table management function unit 10b, and progress bar management function unit 10 c.

The image processing function unit 10 a uses a technique, for example,such as the MPEG4, H.264, or the like, to execute compressionencoding/decompression decoding processing as to moving image contentdata. Also, with the present embodiment, in the case of subjectingmoving image content data already subjected to compression encoding todecompression decoding for the first time, or at the time of subjectingmoving image content data not subjected to compression encoding tocompression encoding, the image processing function unit 10 a detects anobject image from each frame making up this moving image. Note that thedetails of detection processing of this object image will be describedlater.

The table management function unit 10 b generates an object search tablethat correlates the identification information of an object imagedetected by the image processing function unit 10 a, the identificationinformation of a frame including the object image (specifically, framenumber), and playback time information of this frame (timestampinformation), and manages this. Also, in the case that the data of anobject search table has already been accompanied with moving imagecontent data received through the external interface unit 22,short-distance wireless communication unit 23, communication circuit 11,or the like, the table management function unit 10 b manages the movingimage content data and the object search table data in a correlatedmanner. The details of the object search table will be described later.

While moving image content data is played by the image processingfunction unit 10 a, the progress bar management function unit 10 cdisplays a progress bar on the screen of the display unit 13 based onthe total playback time information of the moving image content datathereof, and the playback progress of this moving image content. Notethat the details of the total playback time information, playbackprogress, and progress bar will be described later.

Also, with the present embodiment, in the case that a moving imagecontent being played has been registered in the above object searchtable, the image processing function unit 10 a refers to each of theobject identification information within this table to extract eachframe including an object image corresponding each of the objectidentification information out of the moving image content data thereof.Also, in the case that an object image has been selected from thedisplay image of the moving image content being played, the imageprocessing function unit 10 a may extract a frame corresponding to theobject identification information of this selected object image out ofthis moving image content data by referring to the above object searchtable. Subsequently, the image processing function unit 10 a generates areduced image by resizing the image of this extracted frame. Theprogress bar management function unit 10 c at this time displays thisreduced image in the vicinity of the progress bar on the screen in amanner correlated with the playback execution time of a frame serving asthe basis thereof. Also, at this time, in the case that a reduced imagedisplayed on the screen has been selected, or a playback positioncorresponding to this selected reduced image above the progress bar hasbeen specified, the image processing function unit 10 a moves theplayback position of the moving image content to a frame correspondingto this reduced image. Note that the details of selection of the objectimage, generation of a reduced image, display of this reduced image,movement of a playback position, and so forth will be described later.

The memory unit 15 is made up of built-in memory provided to the insideof this terminal, and detachable card-shaped memory in which so-calledSIM (Subscriber Identity Module) information or the like is stored. Thebuilt-in memory is made up of ROM (Read Only Memory) and RAM (RandomAccess Memory). This ROM stores an OS (Operating System), a controlprogram for the control and arithmetic unit 10 controlling each unit,various types of initialized values, dictionary data, sound data such asringtones, key stroke sounds, and so forth, various types of applicationprograms, and so forth. This ROM includes rewritable ROM such asNAND-type flash memory or EEPROM (Electrically Erasable ProgrammableRead-Only Memory), and is configured so as to save e-mail data, data ofa telephone book or mail address book, data of a still image or movingimage content, data of the object search table, and in addition thereto,various types of user set values, and so forth. The RAM storesoccasional data as a work area or buffer area at the time of the controland arithmetic unit 10 performing various types of data processing.

Note that, with the example in FIG. 1, of storage regions included inthe memory unit 15, only a program storage unit 16 and a data storageunit 17 are illustrated. In the case of the present embodiment, themoving image content data, object search table data, and so forth arestored in the data storage unit 17. Also, in the case of the presentembodiment, a content information processing program for the control andarithmetic unit 10 executing each function of the image processingfunction unit 10 a, table management function unit 10 b, and progressbar management function unit 10 c is also stored in the program storageunit 16 as one of multiple application programs. Note that each of theimage processing function unit 10 a, table management function unit 10b, and progress bar management function unit 10 c may be realized byhardware.

Additionally, the portable telephone terminal according to the presentembodiment also includes components provided to a common portabletelephone terminal such as an LED (Light Emitting Diode) for keylighting or incoming light or the like, and a driving unit thereof, anon-contact communication processing unit including a non-contact ICcard function and a reader/writer function, a GPS (Global PositioningSystem) communication unit, a timer (clock unit), a charge control unit,a battery, and so forth.

Description will be made below regarding detection of an object image,and details of the object search table.

As described above, for example, when subjecting moving image contentdata after compression encoding to decompression decoding for the firsttime, or when subjecting moving image content data before compressionencoding to compression encoding, the image processing function unit 10a detects an object image from a frame making up this moving image.Subsequently, the table management function unit 10 b generates anobject search table that correlates the identification information of anobject image detected from this moving image content with the framenumber including this object image, and playback time informationthereof.

Now, with the present embodiment, the term “object image” means an imagearea to be recognized as a pat of object by an image recognition andanalysis technique within each frame making up a moving image.Specifically speaking, the image processing function unit 10 a detectseach of image areas to be recognized and analyzed as a pat of objectwithin each frame making up a moving image, for example, such as“persons”, “airplanes”, “bicycles”, “trees”, “houses”, and so forth, asan object image. Note that various techniques are already available withthe related art for an image recognition and analysis technique fordetecting an object image, and with an embodiment of the presentinvention any technique may be employed. Also, with the presentembodiment, a frame from which an object image may be detected is asynchronous frame such as an intra frame (I frame) stipulated, forexample, by MPEG4 or H.264. That is to say, a synchronous frame fromwhich an object image is detected is a frame that can make up onedisplay image using only the data of the frame thereof. Note that in thecase that each frame can make up one display image such as a frame ofso-called motion JPEG, the image processing function unit 10 a maydetect an object image from each frame thereof.

Also, with the present embodiment, an object search table OST is, suchas illustrated in FIG. 2, a table that correlates object identificationinformation, a frame number, and timestamp information. This objectsearch table OST is stored in the memory unit 15 in a manner correlatedwith a moving image content file MPv by the table management functionunit 10 b. Note that frames making up a moving image content are arrayedin the order of frame numbers in order of time, and in the case that thetemporal position of each frame within the moving image content can bedetermined by the frame numbers, for example, time information such asthe above timestamp information is not necessarily registered in theobject search table OST.

The above object identification information is information fordetermining each of the object images detected from a moving imagecontent. Also, with the present embodiment, of objects detected from amoving image content, object images recognized as the same object imagesat the time of the above image recognition and analysis are appendedwith the same object identification information. In the case of thepresent embodiment, this object identification information is determinedto be a keyword specifically representing the object thereof as anexample. Specifically, for example, keywords representing objectidentification information are text data such as “person”, “airplane”,“bicycle”, “tree”, “house”, and so forth. It goes without saying thatcommon numerals or codes or the like may be employed as objectidentification information.

The above frame numbers are frame identification information foridentifying each frame within a moving image content, and are numbersassigned to each frame within this moving image content. In the case ofexemplifying a MPEG4 moving image content file as an example, the framenumbers thereof are frame numbers stored in STSS (Sample Table SyncSample atom) stipulated by the MPEG4.

The above timestamp information is, in the case of exemplifying a MPEG4moving image content file as an example, frame playback time informationstored in STSS (Sample Table Sync Sample atom) stipulated by the MPEG4.

Specifically, in the case of exemplifying the MPEG4 as an example, thefile MPv of a moving image content is configured, such as illustrated inFIG. 2, so as to include the above STTS and STSS, and a video frame VF.The above STTS stores the values of a total number of frames, playbacktime of one frame, and a timescale/frame rate, and the above STSS storesthe values of a total number of synchronous frames, and each framenumber. The above video frame VF stores each frame of a moving imagecontent, of which the I frame Fi is a synchronous frame. Thus, with themoving image content file MPv of the MPEG4, various types of videoinformation are managed with atom information, and access to the abovesynchronous frame can be performed by direct access with reference tothe information of “STSSatom”.

Accordingly, in the case of exemplifying the above MPEG4 moving imagecontent, the frame number of the above STSS that enables direct accessto each frame, and the frame playback time of the above STTS astimestamp information that can be used for display corresponding to thetime axis of the progress bar, are stored in the above object searchtable OST. Also, the above object identification information is storedin this object search table OST in a manner correlated with these framenumber and timestamp information. Specifically, such as illustrated inFIG. 2, each frame number where the corresponding object has beendetected, and playback time from the head of the moving image content ateach frame thereof are stored in the object search table OST, andsubsequently, identification information OIa, OIb, and so on of theobjects detected from each frame thereof are stored in a mannercorrelated with the frame number thereof. Note that FIG. 2 illustratesan example wherein object information OIa and OIb are correlated withthe synchronous frame of a frame number “F1” of which the playback timefrom the head of the moving image content is “0 msec”, objectinformation OIb is correlated with the synchronous frame of a framenumber “F2” of which the playback time from the head of the moving imagecontent is “500 msec”, and object information OIa is correlated with thesynchronous frame of a frame number “F3” of which the playback time fromthe head of the moving image content is “100 msec”.

Next, with the portable telephone terminal 1 according to the presentembodiment, a progress bar and reduced images to be displayed on thedisplay screen of the display unit 13 will be described with referenceto FIGS. 3 through 8.

FIG. 3 illustrates a basic display example of the display screen at thetime a moving image content being played.

As illustrated in this FIG. 3, while the moving image content is played,the playback image of the moving image content thereof, and also aprogress bar PB and reduced images RP are displayed.

The progress bar PB has a bar shape that is long in the horizontaldirection of the display screen as an example, and the length in thelateral direction on the screen of the bar thereof corresponds to totalplayback time used for playing this moving image content data from thebeginning to the end. Also, with this progress bar PB, an indicatoraccording to the current playback position of the moving image content,i.e., an indicator Mpb for displaying the playback progress of thismoving image content is displayed. The progress bar management functionunit 10 c creates the display image data of these progress bar PB andthe indicator Mpb based on the above total playback time information,and the information of playback progress. Also, a cursor CP indicatingthe current playback position is displayed on, for example, the leftedge of the indicator Mpb in the drawing. This playback position cursorCP is configured so as to move to an arbitrary position on the progressbar PB by the user. In the case that this playback position cursor CPhas been moved to the user's desired potion on the progress bar PB, theplayback position (playback time) of the moving image content is movedto a position (playback time) corresponding to the playback positioncursor CP after movement thereof.

The reduced images RP are images wherein the image of a framecorresponding to each frame number registered in the object search tableOST is resized and reduced. The reduced images RP are displayed, forexample, in the vicinity of the upper side of the screen of the progressbar PB by being arrayed in the order matched with the playback time ofthe corresponding frame. Note that the display interval on the screen ofthe reduced images RP may be an equal interval, or may be an intervalmatched with the playback time of the corresponding frame. In eithercase, the number of the reduced images RP to be displayed on the screenis determined while taking the size of the screen of the display unit13, and the viewability of the reduced images, into consideration.However, the display order of the reduced images RP is coordinated withthe playback order of the corresponding frame. FIG. 3 illustrates anexample wherein the reproduced images RP are displayed on the screenwith an equal interval.

The image processing function unit 10 a creates the display image dataof the reduced images RP based on the data of the object search tableOST at the time of playback of this moving image content being started.That is to say, the image processing function 10 a creates each reducedimage RP from the frame of each frame number registered in the objectsearch table OST regarding this moving image content. Subsequently, thegenerated reproduced images RP are displayed in the vicinity of theprogress bar PB in an array by the progress bar management unit 10 c.

Now, for example, in the case that all of the reduced images RP are notdisplayable on one screen in an array for the reason that the totalnumber of the reduced images RP is too many, or the like, the imageprocessing function unit 10 a generates only the reduced images RPmatched with predetermined conditions, and the progress bar managementfunction unit 10 c displays the reduced images RP generated based onthese predetermined conditions in the vicinity of the progress bar PB inan array.

With the present embodiment, examples of the above predeterminedconditions include a delimiter point for every certain period, and anoccurrence point of scene change.

Specifically, in the case that a delimiter point for every certainperiod is employed as the above predetermined condition, the imageprocessing function unit 10 a generates only the reduced images RPcorresponding to a frame presented at each delimiter point for everycertain period, and the progress bar management function unit 10 cdisplays these reduced images RP on the screen in an array. Note that inthe case that a frame accurately matched with a condition of a delimiterpoint for every certain period has not been registered in the objectsearch table OST, the image processing function unit 10 a generatesreduced image RP of a frame near that frame registered in the objectsearch table OST.

Also, in the case of employing an occurrence point of scene change asthe above predetermined condition, with the occurrence point of thisscene change, the image processing function unit 10 a generates, forexample, only the reduced image RP corresponding to the first frame ofthis scene. Note that in the case that no frame matched with thecondition of an occurrence point of this scene change has beenregistered in the object search table OST, the image processing functionunit 10 a generates the reduced image RP of a frame registered in theobject search table OST after an occurrence point of this scene change.

Note that the two conditions of the above certain period and scenechange may be employed as a combination of both, or may be employedseparately, and being switched as appropriate by the user's selection orautomatic selection of the terminal or the like. Also, the time intervalof the certain period may be changed as appropriate by the user'sselection or automatic selection of the terminal or the like.

Also, with the present embodiment, the reduced images RP displayed onthe screen such as described above are configured so as to be selectedby the user. With the present embodiment, in the case that one of thereduced images has been selected by the user, the playback position(playback time) of the moving image content is moved to the playbackposition (playback time) of the frame corresponding to the selectedreduced image RP. Note that selection of the reduced image RP at thetime of moving the playback position is performed at the time apredetermined selection instruction such as clicking or the like aftermoving the cursor for selection instruction above the reduced image RP.

FIG. 4 illustrates a specific image display example in the case of amoving image content already registered in the object search table OSTbeing played and displayed such as described above.

With the present embodiment, in the case of the above moving imagecontent being and displayed, the image processing function unit 10 aexecutes detection processing regarding whether or not there is anobject image corresponding to the object identification informationregistered in the object search table OST within the image now beingplayed. Subsequently, upon detecting an object image registered in theobject search table OST within the image now being played, the imageprocessing function unit 10 a adds, such as illustrated in FIG. 4,object detection windows OEa and OEb to object images ObA and ObBdetected within the image now being played, respectively.

The object detection windows OEa and OEb may be a somewhat wide windowso as to cover the detection object images ObA and ObB such asillustrated in the example in FIG. 4, or a window having generally thesame shape and size as those of the detection object images ObA and ObB.These object detection windows OEa and OEb are windows for clearlyindicating that the object images ObA and ObB thereof have been detectedbased on the object search table OST, i.e., are objects registered inthe table OST. With the present embodiment, in order to clearly indicatethat the object images ObA and ObB are images that have been detectedabove, the image processing function unit 10 a subjects the objectwindow detection objects OEa and OEb to predetermined visualizationprocessing. Examples of this predetermined visualization processinginclude processing such as tinting or color change to a window, addingan image frame to a window, enhancement thereof, semi-transparency of awindow, and so forth.

Also, with the present embodiment, the object images ObA and ObB towhich the object detection windows OEa and OEb are added can be selectedby a selection instruction from the user, or a predetermined automaticselection instruction, or the like.

Selection of the object images ObA and ObB by the user's selectioninstruction is, in the case that a touch panel is provided to the wholescreen of the display screen for example, performed when the usertouches the object detection windows OEa and OEb on the screen in FIG.4, or the like. Also, with the present embodiment, selection of theobject images ObA and ObB by the user's selection instruction is alsoperformed when the user directly inputs the above keyword text (objectidentification information) registered in the object search table OSTcorresponding to the object images ObA and ObB thereof. That is to say,the control and arithmetic unit 10 determines that the object images ObAand ObB have been selected by the user at the time of the touch panelarea corresponding to the object detection windows OEa and OEb beingtouched, or at the time of the keyword text corresponding to the objectimages ObA and ObB being input. In addition thereto, an arrangement maybe made wherein attribute information relating to the object images isadded to a keyword, and selection by this attribute can be made insteadof selection by the keyword or along therewith. Also, selection of theobject images by the user's selection instruction may be performed atthe time of a predetermined selection instruction, for example, such asclicking or the like after moving the cursor for selection instructiononto the object detection windows. Further, for example, in the casethat another object image has been registered in the object search tableOST in a manner correlated with the object images ObA and ObB, at thetime of the object images ObA and ObB or the like being selected, thisother object image may be handled to have been selected at the sametime. Note that in the case that object selection is performed bytouching of the object detection windows OEa and OEb such as describedabove, it is desirable that the object detection windows OEa and OEb aresomewhat wide areas. That is to say, according to the somewhat wideareas OEa and OEb, selection of the object images ObA and ObB is readilyperformed.

On the other hand, as for the above automatic selection instruction, itcan be conceived to automatically select the object images ObA and ObBdetected on the screen, or to automatically select, at the time of anobject image already selected and registered being detected, the objectimage thereof, or the like.

With the present embodiment, such as described above, at the time of amoving image content being played and displayed, in the case ofselection of the object images being performed, the image processingfunction unit 10 a generates only the reduced image RP of a frameincluding an object image matched with or relating to the selectedobject image. That is to say, the image processing function unit 10 a atthis time searches for a frame corresponding to the selected objectimage from the object search table OST, and generates only the reducedimage RP of the frame thereof.

Subsequently, the progress bar management function unit 10 c displaysthe reduced images RP generated based on the selected object image inthe vicinity of the progress bar PB in an array. That is to say, inother words, the reduced images RP to be displayed at this time in thevicinity of the progress bar PB in an array are images for informing theuser in which temporal position within the moving image content theframe including the selected object image exists.

Also, with the present embodiment, in order to comprehensively informthe user in which temporal position within the moving image content aframe corresponding to the reduced image RP exists, the progress barmanagement function unit 10 c at this time displays the reduced imagesRP and the playback positions on the progress bar PB in a correlatedmanner.

Description will be made below in detail regarding a display examplewherein the reduced images RP are correlated with the playback positionson the progress bar PB, with reference to FIGS. 5 through 7.

For example, let us say that frames that can be searched from the objectsearch table OST based on the object images ObA and ObB in FIG. 4 aresix search frames SF1 through SF6 such as illustrated in (a) through (f)in FIG. 5. Specifically, let us say that there are search frames SF1,SF2, SF3, and SF6 in FIG. 5 as frames that can be searched from theobject search table OST based on the object ObA, and similarly, thereare search frames SF1, SF3, SF4, and SF5 in FIG. 5 as frames that can besearched from the object search table OST based on the object ObB. Notethat, with the present embodiment, for example, even though the sizes ofimages, and image components are not completely matched and somewhatdiffer such as object images ObA1, ObA2, and ObA3 in FIG. 5, in theevent that these are recognized to be the same objects, these arehandled as the same objects. Description will be made later regardingthe details of processing at the time of determining, even though thesizes and components of images somewhat differ, these to be the sameobjects.

Now, for example, if we say that the object image ObA has been selectedsuch as illustrated in FIG. 6, the table management function unit 10 bdetermines the frames SF1, SF2, SF3, and SF6 by searching for the objectsearch table OST based on the identification information of the objectimage ObA. The image processing function unit 10 a at this timegenerates the reduced images RP1, RP2, RP3, and RP6 corresponding to theframes SF1, SF2, SF3, and SF6, such as illustrated in FIG. 6. Also, theprogress bar management function unit 10 c displays the reduced imagesRP1 through RP3, and RP6 in the vicinity of the progress bar PB in anarray.

Similarly, for example, if we say that the object image ObB has beenselected such as illustrated in FIG. 7, the table management functionunit 10 b determines the frames SF1, SF3, SF4, and SF5 by searching forthe object search table OST based on the identification information ofthe object image ObB. The image processing function unit 10 a at thistime generates the reduced images RP1, RP3, RP4, and RP5 correspondingto the frames SF1, SF3, SF34 and SF5 such as illustrated in FIG. 7.Also, the progress bar management function unit 10 c displays thereduced images RP1, RP3 through RP5 in the vicinity of the progress barPB in an array.

Also, at this time, the progress bar management function unit 10 crecognizes in which temporal position within the moving image contentthe frames SF exist, based on the timestamp information of the objectsearch table OST, and displays the reduced images RP in a mannercorrelated with the temporal positions. As an example, such asillustrated in FIGS. 6 and 7, the progress bar management function unit10 c displays time point marks TP representing the playback point intime of the frame SF corresponding to a reduced image RP on the progressbar PB. Also, the progress bar management function unit 10 c alsodisplays connecting line marks CL for connecting a time point mark TP onthe progress bar PB, and the corresponding reduced image RP. That is tosay, in the case of the example in FIG. 6, time point marks TP1, TP2,TP3, and TP6 representing the playback points in time of the frames SF1through SF3, and SF6 corresponding to the reduced images RP1 throughRP3, and RP6 respectively are displayed, and also connecting line marksCL1, CL2, CL3, and CL6 for connecting the time point marks TP1 throughTP3, and TP6, and the reduced images RP1 through RP3, and RP6respectively are displayed. Similarly, in the case of the example inFIG. 7, the time point marks TP1, and TP3 through TP5 corresponding tothe reduced images RP1, and RP3 through RP5 respectively are displayed,and also the connecting line marks CL1, and CL3 through CL5 aredisplayed. In the case of the present embodiment, the colors and designsof the time point marks and connecting line marks are configured so asto be visually understandable.

Also, in the case that all of the reduced images RP are not displayabledue to a reason that the number of frames searched from the objectsearch table OST based on the selected object image is too many, or thelike, the progress bar management function unit 10 c displays detectionframe marks TF representing the temporal position of a frame of whichthe reduced image is not displayable, on the progress bar PB. Thesedetection frame marks TF are represented with multiple line displayscorresponding to the temporal positions of frames, and are in the casethat frames are close in time, represented with area display includingthese adjacent frames, or the like. That is to say, in the case of theexample in FIG. 6, the detection frame marks TF1, TF2, TF3, and TF6 aredisplayed on the progress bar PB to display a frame of which the reducedimage RP is not displayable. Similarly, in the case of the example inFIG. 7, the detection frame marks TF1, TF3, TF4, and TF5 are displayedon the progress bar PB to display a frame of which the reduced image RPis not displayable. Note that in the case that the area displayincluding the adjacent frames is employed, when there is a portion whereframes are crowded in time, or a temporally sparse portion within thearea thereof, such a crowded or sparse portion may be represented withcolored gradation or the like, for example. Specifically, for example,the crowded portion is represented with a deep color, and the sparseportion is represented with a light color. Also, in such a case whereall of the reduced images RP are not displayable, frames representedwith the time point marks TP, i.e., frames representing the reducedimages RP may be determined, in the same way as described above, basedon a predetermined condition such as a delimiter point for every certainperiod, an occurrence point of scene change, or the like.

Note that movement of the playback position due to movement of theplayback position cursor CP may be performed at the time of the playbackposition cursor CP being moved above the user's desired time point markTP or detection frame mark TF. That is to say, in this case, theplayback position of the moving image content is moved to the playbackposition corresponding to the time point mark TP or detection frame markTF specified by the playback position cursor CP.

With the present embodiment, the examples in FIGS. 6 and 7 illustrate acase where a desired object image has been selected from the playbackimages of the moving image content, but the number of objects that canbe selected from this playback image is not restricted to one.

FIG. 8 illustrates a display example in the case that two of the objectimages ObA and ObB have been selected out of the playback images of themoving image content, for example.

That is to say, such as illustrated in the example in FIG. 8, in thecase that the object images ObA and ObB have been selected, the tablemanagement function unit 10 b determines the frames SF1 through SF6 inFIG. 8 by searching for the object search table OST based on theidentification information of the object images ObA and ObB. The imageprocessing function unit 10 a in this case generates, such asillustrated in FIG. 8, reduced images RP1 through RP6 corresponding tothe frames SF1 through SF6. Also, the progress bar management functionunit 10 c displays these reduced images RP1 through RP6 in the vicinityof the progress bar PB in an array. Also, in the case of the example inFIG. 8 as well, in the same way as described above, the time point marksTP1 through TP6 and connecting line marks CL1 through CL6 correspondingto the reduced images RP1 through RP6 are displayed above the progressbar PB. Further, in the same way as described above, in the case thatthe number of frames searched from the object search table OST based onthe selected object images ObA and ObB is too many, and all of thereduced images RP are not displayable, the detection frame marks TF1through TF6 are displayed on the progress bar PB.

Next, description will be made regarding processing at the time of theimage processing function unit 10 a determining, even though the sizesof object images, or components making up the object images, or the likesomewhat differ, these objects to be the same object image, withreference to FIGS. 9 through 12.

A moving image content is configured of multiple temporally consecutiveframes. Subsequently, the playback display of this moving image contentis performed by displaying each frame thereof in order of time, andaccording to the direction and size of an object image within each frameto be displayed in order of time gradually changing, movement of such anobject is expressed on the display screen.

With the image recognition and analysis technique according to therelated art, for example, in the case that there are several objectimages represented with the same “person” or the same “airplane” or thelike, at the time of the size of each object image, the shape, position,number or the like of a component of each object image changing, theseare detected as different images.

That is to say, for example, such as illustrated in (a) through (i) inFIG. 9, in the case that the object image ObA representing the sameperson's face is included in each frame, at the time of both imagesdiffering such as object images ObA1 and ObA4, the image recognition andanalysis technique according to the related art recognizes the objectimages ObA1 and ObA2 to be different images. Similarly, at the time ofboth images differing such as object images ObA2 and ObA3, the imagerecognition and analysis technique according to the related artrecognizes the object images ObA2 and ObA3 to be different images. Also,for example, in the event that the number of image portions of “eye”that is one of the components of an object image differs, the imagerecognition and analysis technique according to the related artrecognizes the object images ObA1 and ObA2 to be different images. Thatis to say, in the case of the example in FIG. 9, regardless of theobject images ObA1 through ObA4 being object images representing thesame person's face, the image recognition and analysis techniqueaccording to the related art recognizes the object images to bedifferent images.

In order to handle such a situation, the portable telephone terminal 1according to the present embodiment can determine, even though thesizes, components, or the like of multiple object images representingthe same “person” or the like differ, these to be the same object imageby taking advantage of a motion prediction technique for moving images.

Therefore, the image processing function unit 10 a of the control andarithmetic unit 10 also detects, at the time of detection of an objectimage, each component (hereafter, referred to as “sub object image”)making up the object image thereof.

For example, such as illustrated in (a) in FIG. 10, in the event ofhaving detected an object image ObAp from a certain processing objectframe Fp image, the image processing function unit 10 a also detects subobject images Sp1 through Sp4 making up the object image ObApsimultaneously. Also, such as illustrated in (b) in FIG. 10, in theevent of having detected an object image ObAc from the next processingobject frame Fc, the image processing function unit 10 a also detectssub object images Sp1, Sp3, and Sp4 making up the object image ObAcsimultaneously. Subsequently, such as illustrated in (b) in FIG. 10, theimage processing function unit 10 a confirms, based on motion vectorinformation from the object image ObAp to the object image ObAc betweenboth processing object frames Fp and Fc, the movement destination of thesub object images Sp1 through Sp4 of the object image ObAp.

Subsequently, for example, in the event that all of the sub objectimages are assembled of the object images of both processing objectframes, even though the sizes of the object images of both processingobject frames differ for example, the image processing function unit 10a according to the present embodiment subsequently handles these as thesame object image. Also, with the object images of both processingobject frames, even though a portion of each sub object image iseliminated or increased, and the shapes thereof change, in the eventthat the amount of the elimination or increase, and shape change isincluded in a predetermined certain allowable range, the imageprocessing function unit 10 a handles these as the same object image. Inthe case of the example in FIG. 10, the sub object image of the objectimage ObAc of the processing object frame Fc is partially eliminated ascompared to the object image ObAp of the processing object frame Fp, butthe number of eliminations is included in a predetermined certainallowable range. Accordingly, the image processing function unit 10 ahandles the object images ObAp and ObAc as the same object image.

Specifically, with the present embodiment, for example, such asillustrated in (a) through (i) in FIG. 11, the image processing functionunit 10 a confirms, based on the motion vector information of the objectimages ObA detected within processing object frames arrayed in order oftime for example, the movement destination of the subject images withinthe object images ObA. Subsequently, in the event that the change amountof the number or shape or the like of sub object images is included in acertain allowable range of the object images ObA of the processingobject frames, such as illustrated in (a) through (i), the imageprocessing function unit 10 a handles the object images ObA of theprocessing object frames as the same object image.

Thus, the control and arithmetic unit 10 according to the presentembodiment confirms, based on the motion vector of an object image, themovement destination of the sub object images, and also has enhancedtolerance as to change in the size or components of the object image bysetting a certain allowable range as to change in the sub object imagesthereof.

Note that the control and arithmetic unit 10 according to the presentembodiment is configured, as described above, so as to determinemultiple object images of which the sizes or a part of components or thelike differ to be the same object image. Accordingly, for example, atthe time of displaying reduced images in the vicinity of the progressbar in an array, reduced images including object images of which thesizes or components somewhat change may be arrayed instead of arrayingreduced images including object images of which the sizes or componentsare completely the same.

FIG. 13 illustrates a processing flow at the time of the control andarithmetic unit 10 executing detection of an object, creation of anobject search table, and generation and display of reduced images and aprogress bar in the event that decoding of a moving image content isperformed at the portable telephone terminal 1 according to theembodiment of the present invention.

In FIG. 13, first, as step S1, when the image processing function unit10 a starts playback of a moving image content, the table managementfunction unit 10 b determines as processing in step S2 whether or notthe object search table has already been created as to the moving imagecontent thereof. Subsequently, the table management function unit 10 binforms the image processing function unit 10 a of the determinationresult thereof.

Upon receiving notice to the effect that the object search table has notbeen created from the table management function unit 10 b, the imageprocessing function unit 10 a proceeds to step S3 to start decodingprocessing of the moving image content. Also, upon receiving notice tothe effect that the object search table has been created from the tablemanagement function unit 10 b, the image processing function unit 10 aproceeds to step S10 to start decoding processing of the moving imagecontent.

In the case of having proceeded to the processing in step S3 to startdecoding processing, the image processing function unit 10 a determinesas processing in step S4 whether the decoded frame is the synchronousframe or a frame other than the synchronous frame. Specifically, forexample, in the case of decoding by the MPEG4, the image processingfunction unit 10 a determines whether the decoded frame is an I frame orP (or B) frame.

Subsequently, in the event that determination is made in step S4 thatthe decoded frame is a synchronous frame (I frame), the image processingfunction unit 10 a advances the processing to step S5, detects an objectimage such as described above, and advances the processing to step S7.On the other hand, in the event that determination is made that thedecoded frame is not a synchronous frame (P (or B) frame), the imageprocessing function unit 10 a advances the processing to step S6 toperform tracing of an object based on a motion prediction, and advancesthe processing to step S7.

Next, the table management function unit 10 b uses the object image thatthe image processing function unit 10 a detected in step S5 to updatethe object search table. That is to say, the table management functionunit 10 b adds table data corresponding to the moving image contentbeing decoded to the existing table management function unit.

Subsequently, as processing in step S8, the progress bar managementfunction unit 10 c displays the moving image of the moving image contenton the screen of the display unit 13. Note that at this time theprogress bar alone is displayed, but the reduced images are notdisplayed.

Subsequently, as the processing in step S9, the image processingfunction unit 10 a determines whether or not decoding of the movingimage content has been completed. Subsequently, in the case thatdecoding of the moving image content has not been completed, the controland arithmetic unit 10 returns the processing to step S3, and in thecase that decoding of the moving image content has been completed,advances the processing to step S13.

Upon advancing to the processing in step S13, the table managementfunction unit 10 b completes creation of the object search tablecorresponding to the moving image content. After this step S13, thecontrol and arithmetic unit 10 ends the processing of this flowchart instep S14.

Also, in the event that determination is made in step S2 that the objectsearch table has been created, and the processing proceeds to step 10,the image processing function unit 10 a executes decoding of the movingimage content, and also creates a reduction image such as describedabove based on the object search table and the selection instruction ofan object image by the user or the like.

Also, as the processing in step S11, the progress bar managementfunction unit 10 c at this time displays the progress bar such asdescribed above, and displays the reduced images in the vicinitythereof.

Subsequently, as the processing in step S12, the image processingfunction unit 10 a determines whether or not decoding of the movingimage content has been completed. Subsequently, in the event thatdecoding of the moving image content has not been completed, the controland arithmetic unit 10 returns the processing to step S10, and in theevent that decoding has been completed, advances the processing to theabove step S14.

The following examples can be conceived as other embodiments of thepresent invention.

With the above progress bar, a portion thereof may be enlarged anddisplayed. In this case, with regard to each frame corresponding to theprogress bar of this enlarged and displayed portion, detection of anobject image such as described above, reference of the object searchtable by the detected object image, display of reduced images based onreference thereof, or the like may be performed.

The reduced images displayed in the vicinity of the progress bar in anarray may have the same size, or may have a different size. That is tosay, as an example, the reduced image of a frame nearest the frame beingnow played may be displayed greater than others, or may be changed incolor.

The reduced images displayed in the vicinity of the progress bar in anarray may be, for example, semi-transparent images so as not to hinderthe images of the moving image content being played.

The reduced images arrayed in the vicinity of the progress bar may beonly reduced images of frames that are portions that have not beenplayed. That is to say, each time the playback display of each frameends, the reduced image of the frame after playback display maysequentially be eliminated from the screen. Thus, in the case that thereduced image of a frame after playback display is sequentiallyeliminated from the screen, many more screen display regions can besecured as to each reduced image of a frame that has not been played,and accordingly, the number of displays of reduced images of framesafter playback display can be increased, for example. In goes withoutsaying that the reduced image of a frame after playback display may beremained on the screen as is. Thus, in the case that the reduced imageof a frame after playback display has been remained on the screen, theuser can readily confirm a desired frame after playback display, and canreadily specify, for example, a frame serving as a playback object atthe time of so-called reverse playback.

The object identification information registered in the object searchtable may be liked with, for example, address book data within theportable telephone terminal. For example, in the case that the objectidentification information is correlated with a name or face image orthe like registered in the address book data, the user can specify aname or face image of the address book by selecting an object image.That is to say, for example, in the case that there is the object imageof a person's face registered in the address book within the playbackimage of the moving image content, access as to this person by telephoneor mail can be performed by selecting the object image of the facethereof.

With the above embodiment, in the event that an object image has beenselected during playback of a moving image content, the framecorresponding to this selected object image is searched, but the frameto be searched based on this selected object image may be a frame withinanother moving image content.

CONCLUSION

As described above, according to the portable telephone terminalaccording to an embodiment of the present invention, reduced images offrames including a desired object image are displayed above the progressbar during playback display of a moving image content. Also, accordingto the portable telephone terminal according to the present embodiment,in the case that there is an object image registered in the objectsearch table within in an image being played and displayed, the objectimage thereof can be selected. With the portable telephone terminalaccording to the present embodiment, in the case that a desired objectimage has been selected, reduced images of frames including the selectedobject image thereof are displayed. Accordingly, the user using theportable telephone terminal according to the present embodiment can finda desired scene including a desired object image in a short period oftime, readily and in a sure manner. In particular, in the case that amoving image is displayed on the screen of the portable telephoneterminal, the user is not able to continuously gaze at the screen, andaccordingly, it is difficult to fine a desired scene, for example, byfast forwarding or fast rewinding, but according to the presentembodiment, the user does not have to perform gazing at the screen byfast forwarding or fast rewinding, and can speedily find the desiredscene.

Also, according to the present embodiment, detection of an object image,and generation of a reduced image are performed regarding a synchronousframe that can be decoded with only one frame, and reference of otherframes does not have to be performed. Therefore, with the presentembodiment, image processing for detection of an object image and forgeneration of a reduced image is light, and accordingly, detection of anobject image, and generation of a reduced image can speedily beperformed. Also, for example, updating of the display can be performedin a short period of time even at the time of updating the display ofreduced images by selection of an object image.

Note that description of the above embodiment is an example of thepresent invention. Accordingly, the present invention is not restrictedto the above embodiment, and it goes without saying that various changescan be made according to a design or the like without departing from theessence and spirit of the present invention.

Though the above embodiment assumes a portable telephone terminal, thepresent invention may also be applied to video cameras or digitalcameras whereby playing and taking of a moving image can be performed,and personal digital assistants such as portable personal computers.

1. (canceled)
 2. An information processing apparatus comprising: areproduction circuit configured to reproduce video content comprising aplurality of frames; and a processing circuit configured to receive, viaa user interface, a user selection of one or more keywords; extract aframe including an object image from the video content according to theuser selection such that when the user selection comprises more than onekeyword, a plurality of keywords corresponding to object identificationinformation of the object image in the extracted frame correspond tosaid more than one keyword in the user selection; and generate displaydata of a reduced image corresponding to the frame for display.
 3. Theinformation processing apparatus according to claim 2, wherein theprocessing circuit is configured to detect the object imagecorresponding to the object identification information from the framewhen the video content is reproduced.
 4. The information processingapparatus according to claim 2, wherein the processing circuit isconfigured to generate a progress bar displaying a total playback timeand a playback progress of the video content.
 5. The informationprocessing apparatus according to claim 4, wherein the processingcircuit is configured to populate the progress bar with a predeterminedmark correlating the reduced image with the progress bar based onplayback time information of the frame.
 6. The information processingapparatus according to claim 2, wherein the processing circuit isconfigured to move a reproduction position of the video content to areproduction position of the frame.
 7. The information processingapparatus according to claim 2, wherein the processing circuit isconfigured to detect the object image from each of the plurality offrames of the video content during reproduction to generate the objectidentification information and the plurality of keywords.
 8. Theinformation processing apparatus of claim 2, wherein the processingcircuit is further configured to display, via the user interface, anobject detection window displaying a plurality of object images forselection, receive an input corresponding to one of the plurality ofdisplayed object images, and extract a frame from the plurality offrames including the selected object image, and generate the displaydata including the extracted frame.
 9. The information processingapparatus of claim 2, wherein the processing circuit is configured toextract, of a plurality of frames including the object image, a singleframe representative of the plurality of frames over a predeterminedtime period to generate the display data.
 10. The information processingapparatus of claim 2, wherein the processing circuit is configured toextract, of a plurality of frames including the object image, a frame ata time of a scene change occurring from the video content to generatethe display data.
 11. The information processing apparatus of claim 2,wherein the processing circuit is configured to generate the displaydata to further include a bar that displays a playback timecorresponding to a sequence of frames from the video content thatinclude the object image, the sequence of frames beginning with theframe.
 12. The information processing apparatus according to claim 2,wherein the object identification information is linked with addressbook data indicating a name or face image registered in the address bookdata.
 13. A non-transitory computer-readable recording medium includingcomputer program instructions, which when executed by an informationprocessing apparatus, cause the information processing apparatus toperform a method comprising: reproducing video content comprising aplurality of frames; receiving, via a user interface, a user selectionof one or more keywords; extracting a frame including an object imagefrom the video content according to the user selection such that whenthe user selection comprises more than one keyword, a plurality ofkeywords corresponding to object identification information of theobject image in the extracted frame correspond to said more than onekeyword in the user selection; and generating display data of a reducedimage corresponding to the frame for display.
 14. The non-transitorycomputer-readable recording medium according to claim 13, wherein thecomputer program instructions further comprise computer programinstructions for: detecting the object image corresponding to the objectidentification information from the frame when the video content isreproduced.
 15. The non-transitory computer-readable recording mediumaccording to claim 13, wherein the computer program instructions furthercomprise computer program instructions for: generating and displaying aprogress bar indicating a total playback time and a playback progress ofthe video content.
 16. The non-transitory computer-readable recordingmedium according to claim 15, wherein the computer program instructionsfurther comprise computer program instructions for: populating theprogress bar with a predetermined mark correlating the reduced imagewith the progress bar based on playback time information of the frame.17. The non-transitory computer-readable recording medium according toclaim 13, wherein the computer program instructions further comprisecomputer program instructions for: receiving, via the user interface, aselection corresponding to the reduced image; and moving a reproductionposition of the video content to a reproduction position of the framecorresponding to the selected reduced image.
 18. The non-transitorycomputer-readable recording medium according to claim 13, wherein thecomputer program instructions further comprise computer programinstructions for: displaying an object detection window including aplurality of object images for selection; receiving an inputcorresponding to one of the plurality of displayed object images;extracting a frame from the plurality of frames including the selectedobject image; and generating the display data of said reduced image fromthe extracted frame.
 19. The non-transitory computer-readable recordingmedium according to claim 13, wherein the computer program instructionsfurther comprise computer program instructions for: extracting, of aplurality of frames including the object image, a single framerepresentative of the plurality of frames over a predetermined timeperiod to generate the display data.
 20. The non-transitorycomputer-readable recording medium according to claim 13, wherein thecomputer program instructions further comprise computer programinstructions for: extracting, of a plurality of frames including theobject image, a frame at a time of a scene change occurring from thevideo content to generate the display data.
 21. The non-transitorycomputer-readable recording medium according to claim 13, wherein thecomputer program instructions further comprise computer programinstructions for: generating the display data to further include a barthat displays a playback time corresponding to a sequence of frames fromthe video content that include the object image, the sequence of framesbeginning with the frame.